The current state-of-the-art for video encoding is the ITU-T H.265/HEVC video coding standard. It defines a number of different profiles for different applications, including the Main profile, Main Still Picture profile and others. There are a number of standards for encoding/decoding images and videos, including H.265, that use block-based coding processes. In these processes, the image or frame is divided into blocks, with sizes typically ranging from 4×4 to 64×64, although non-square blocks may be used in some cases, and the blocks are spectrally transformed into coefficients, quantized, and entropy encoded. In many cases, the data being transformed is not the actual pixel data, but is residual data following a prediction operation. Predictions can be intra-frame, i.e. block-to-block within the frame/image, or inter-frame, i.e. between frames (also called motion prediction).
When spectrally transforming residual data, many of these standards prescribe the use of a discrete cosine transform (DCT) or some variant thereon. The resulting DCT coefficients are then quantized using a quantizer to produce quantized transform domain coefficients. The blocks of quantized coefficients are then entropy encoded and packaged with side information, like motion vectors and other data, to produce a bitstream of encoded video.
At the decoder, the bitstream is entropy decoded to reconstruct the quantized coefficients. The decoder then inverse quantizes and inverse transforms the reconstructed quantized coefficients to reconstruct the pixel domain residual. Using the same prediction operation as was used at the encoder, the pixel data is then reconstructed.
When encoding quantized coefficients, a number of syntax elements may be used to signal the magnitude of any non-zero coefficients, including significant-coefficient flags, greater-than-one flags, greater-than-two flags, and further level data. A sign bit is also usually encoded and decoded to indicate whether the coefficient is positive or negative. The coding of the sign bit is not context-based since the occurrence of positive and negative signs is usually statistically about equal.
HEVC, and some other video coding schemes, allow for “transform skipping” in some situations. Transform skipping refers to “skipping” the DCT operation and simply quantizing the actual pixel-domain residual data as though it was coefficient data, and then encoding that quantized data. It may also be considered to be application of an identity transform, as opposed to a DCT. In another situation, the transform operation is skipped when lossless coding is used (in which both transform and quantization are skipped). Transform skipping in any of these situations results in data that does not necessarily conform to the assumptions underlying the design of some elements of the encoding process in HEVC.
Similar reference numerals may have been used in different figures to denote similar components.